Comparing Observed Bug and Productivity Rates for Java and C++

An experiment was conducted to compare programmer productivity and defect rates for Java and C++. A modified version of the Personal Software Process (PSP) was used to gather defect rate, bug rate, and productivity data on C++ and Java during two real world development projects. A bug is defined to be a problem detected during testing or deployment. A defect is either a bug, or an error detected during compile time. A typical C++ program had two to three times as many bugs per line of code as a typical Java program. C++ also generated between 15 per cent and 50 per cent more defects per line, and perhaps took six times as long to debug. Java was between 30 per cent and 200 per cent more productive, in terms of lines of code per minute. When defects were measured against development time, Java and C++ showed no difference, but C++ had two to three times as many bugs per hour. Statistics were generated using Student’s t-test at a 95 per cent confidence level. Some discussion of why the differences occurred is included, but the reasons offered have not been tested experimentally. The study is limited to one programmer over two projects, so it is not a definitive experimental result. The programmer was experienced in C++, but only learning Java, so the results would probably favour Java more strongly for equally-experienced programmers. The experiment shows that it is possible to experimentally measure the fitness of a programming language. Copyright  1999 John Wiley & Sons, Ltd.